perm filename DOLPH.TIM[TIM,LSP] blob
sn#616169 filedate 1981-09-30 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00003 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00002 00002 ∂15-Sep-81 0015 the tty of Geoffrey S. Goodfellow Dolphin/KL-10 Benchmark (from SUMEX bboard).
C00009 00003 ∂22-Sep-81 1137 Doug Appelt <APPELT at SRI-AI> Re: Don't
C00013 ENDMK
C⊗;
∂15-Sep-81 0015 the tty of Geoffrey S. Goodfellow Dolphin/KL-10 Benchmark (from SUMEX bboard).
Date: 15 Sep 1981 0011-PDT
Sender: GEOFF at SRI-CSL
Subject: Dolphin/KL-10 Benchmark (from SUMEX bboard).
From: the tty of Geoffrey S. Goodfellow
Reply-To: Geoff at SRI-CSL
To: DHare, Moriconi, Shostak, Levitt, JGOLDBERG
Cc: rpg at SAIL
Message-ID: <[SRI-CSL]15-Sep-81 00:11:18.GEOFF>
Date: 1 Sep 1981 1124-PDT
From: RINDFLEISCH@SUMEX-AIM
Subject: DOLPHIN/KL-10 BENCHMARK
To: Bboard
24-Aug-81 12:47:52-PDT,3542;000000000001
Mail-from: ARPANET host PARC-MAXC rcvd at 24-Aug-81 1247-PDT
Date: 24 Aug 1981 12:47 PDT
Sender: BURTON at PARC-MAXC
to: feigenbaum@sumex-aim, rindfleisch@sumex-aim
subject: Dolphin Interlisp KL-ONE benchmark
from: The Interlisp-D group (Xerox Palo Alto Research Center)
reply-to: burton
We have often been asked to summarize the overall performance of Dolphin
Interlisp relative to other systems used for AI research. However, we have foun
**d
that comparisons based on isolated single measurements (of, for example, the
number of Lisp "instructions" per second or the times taken for function call or
examples such Ackerman's function, list traversal, etc. etc.) vary wildly,
depending on what aspects of the two performance profiles they tap. It is in
response to this variability that we have characterized the Dolphin's
performance, averaged across a wide variety of measurements, in terms of KA-10
equivalents, rather than in terms of individual benchmarks.
Nevertheless, continued concern as to the adequacy of Dolphin Interlisp for
serious AI research has persuaded us that some specific demonstration of its
performance would be useful. In light of the variability of small benchmarks,
we decided to measure a large, existing Interlisp AI system running on both a
Dolphin (provided by XEOS) and a DEC KL-10. In addition to measuring a
broader range of system behavior (including swapping and garbage collection),
we felt that such a benchmark would be more representative of the
computational loads encountered in AI research.
Bill Mark and Tom Lipkis of ISI were kind enough to help us by carrying out
timing comparisons of the part of their Consul system, written in KL-ONE,
which classifies concepts into a KL-ONE network. KL-ONE is a knowledge
representation formalism developed in Interlisp at BBN which is becoming
increasingly popular in the AI community. The same source code was compiled
and run unchanged in normal production versions of both systems. No special
optimizations were done, nor were any standard system facilities (such as
garbage collection) disabled in either system.
Timings were done under a variety of load conditions on the KL-10. The load
average (LA) during t↓u test was monitored approximately once a minute. Times
given are elapsed time, as measured with a stopwatch to guard against
unreported overhead. All times are in seconds.
elapsed total cpu gabage collection cpu less GC
Dolphin 145 139 included in cpu -
KL-10 (LA .2 - .8) 84 64.5 25.5 39
KL-10 (LA .6 - 1.0) 116 75 31 45
KL-10 (LA 1.1 - 2.1) 265 84 47 47
KL-10 (LA 2.3 - 3.5) 415 90 42 48
KL-10 (LA 4.6 - 11.3) 905 86 38 48
The dramatic variation in elapsed time indicates how misleading measures of
system internal clocks can be as measures of delivered computing power. In
terms of what the user gets, these measures indicate that a Dolphin delivers, fo
**r
this benchmark, computing power roughly comparable to that delivered by a
KL-10 with a load average of ~1.5. Further, although these figures indicate tha
**t
Dolphin Interlisp already dominates Interlisp on a KL-10 under normal operating
conditions, we expect further significant performance improvement over the next
two months.
-------
∂22-Sep-81 1137 Doug Appelt <APPELT at SRI-AI> Re: Don't
Date: 22 Sep 1981 1131-PDT
From: Doug Appelt <APPELT at SRI-AI>
Subject: Re: Don't
To: RPG at SU-AI
In-Reply-To: Your message of 22-Sep-81 1005-PDT
Dick,
Thanks for the reminder. Here it is:
Gents: the following is the timings on a run by a NL phd thesis program
on a fairly large example, exercising many parts of InterLisp, especially
spaghetti. Apparently stack fragmentation is quite a problem according to
my source:
LISP TIMING STATISTICS ON SAMPLE PROBLEM
Fast Dorado Microcode
Trial 1 Trial 2 Trial 3 Trial 4 Avg.
-------------------------------------------------------------------------------
Elapsed time 104.96 278.33 159.20 113.05 163.88
SWAP time 18.82 15.89 21.47 21.36 19.38
CPU Time 86.14 262.44 137.73 91.69 144.50
Page Faults 636 687 853 945 780
Swap writes 324 157 220 150 213
Normal Dorado Microcode
Trial 1 Trial 2 Trial 3 Trial 4 Avg.
-------------------------------------------------------------------------------
Elapsed time 157.04 160.15 160.99 166.49 161.17
SWAP time 17.74 13.58 19.94 24.00 18.81
CPU time 139.29 146.57 141.05 142.49 142.36
Page Faults 631 664 820 939 763
Swap Writes 330 117 205 246 224
Ordinary Dolphin
Trial 1 Trial 2 Trial 3 Trial 4 Avg.
-------------------------------------------------------------------------------
Elapsed Time 802.01 857.71 829.86
SWAP time 91.82 109.08 100.45
CPU time 710.19 748.63 729.41
Page Faults 1551 1841 1696
Swap Writes 502 537 519
Production Dolphin
Timings have been adjusted by a factor of .8988764 to compensate for fast clock
Trial 1 Trial 2 Trial 3 Trial 4 Avg.
-------------------------------------------------------------------------------
Elapsed Time 686.89 703.90 688.21 697.77 694.19
SWAP time 19.72 17.05 23.10 30.18 22.51
CPU time 667.17 686.85 665.11 667.59 671.68
Page Faults 372 494 574 638 519
Swap Writes 173 12 90 108 96
INTERLISP-10 on SRI-AI 2060
Trial 1 Trial 2 Trial 3 Trial 4 Avg.
-------------------------------------------------------------------------------
CPU Time 65.01 66.33 61.21 61.90 63.61
GC Time 6.82 11.21 8.55 16.30 10.72
-------